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ABSTRACT 

This paper describes the development of a 
questionnaire for evaluating the activities of the Employee 
Development Program (EDP) at Paradise Valley Community College Center 
(PVCCC) in Phoenix (Arizona) • Four major goals of the evaluation of 
the activities of the EDP, and a means for ensuring the content 
validity of the questionnaire are described. Statistical procedures 
for analyzing empirical data to be generated from the future 
administration of the questionnaire, and a procedure for scanning 
empirical data into an ASCII computer data file and for downloading 
the file into an IBM-PC are identified. Specific procedures are 
identified for reporting and analyzing written comments that ar^ 
generated in response to the questionnaire, and recommendations are 
made regarding procedures used for administering the questionnaire. 
The questionnaire, which evaluates t *e effectiveness of employee's 
participation in the activities of the EDP in terms of helping them 
to achieve the goals of the EDP, was developed based on the goals of 
the evaluation, a review of the related literature, and advice from 
three research experts. The questionnaire can be revised yearly, and 
it identifies the percentage of employee participation in each of the 
employee development activities listed in the questionnaire, new 
activities that would potentially result in the personal and 
professional development of the participants, and suggestions for 
improving the questionnaire's design and administration. A general 
purpose data sheet is included. (RLC) 
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ABSTRACT 



A study of the develODment, analysis, and administration of questionnaires was conducted. 
As a result of this study, an extensivo questionnaire was developed to evaluate the activities 
of the Employee Development Program at Paradise Valley Community College Center. 
Specifically, tne goals of the future evaluation of the activities of the Employee Development 
Program were deterrmned, and a means for ensuring the content validity cf the questionnaire 
was developed and implemented. In addition, statistical procedures were chosen for the analysis 
of the empirical data that will be generated from the future implementation of the questionnaire. 
Moreover, a procedure for scanning empirical data into an ASCII computer data file and for 
downloading the file into an IBM-PC was identified. The ABstat version 6.02 computer 
software statistical program (AQslaL 1989) was suggested for use in analyzing the data 
generated from the future administration of the questionnaire. Futhermore, specific procedures 
were identified for the reporting and analysis of any written comments that may be generated 
from the future administration of the questionnaire. In addition, specific recommendaoons 
were made regarding the procedures to be used for the future administration of the 
questionnaire. Lastly, the questionnaire was field tested. 

Based on the results of this study, the recommendation was made that the Provost of Paradise 
Valley Community College Center charge the Employee Development Committee members with 
the task of administering and analyzing the questionnaire developed for this study. Based on the 
analysis of the data generated from the tuture administration of the questionnaire, the Employee 
Development Committee members will have data upon which to base recommendations for the 
improvement of the Employee Devebpment Program. 
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INTRODUCTION 



Effective employee development programs are characteristic of well-run institutions of 
higher education (Roueche and Baker, 1983). Yet, it is important that employees' participation 
in employee development activities do, in fact, result in the personal and professional 
development of the participants. If not, employees will resent administrators who espouse 
lofty employee development goals but do not put them into practice (Jaffa and Scott, 1988). 
Paradise Valley Community College Center's Emptoyee Development Program is designed to 
**fbster personal and professional development among all college employees** fEmployee 
Development: A Statement r» Philosophy brochure (n.d.:1). The four goals of the Employee 
Development Program at Paradise Valley Community College Center as stated in the Employee 
Development: A Statement of Philosophy brochure (n.d.:3) are as follows: 

1 . To give employees an opportunity to understand the mission of the institution and their 
role. 

2. To help employees improve their job performance in terms of effectiveness, efficiency, 
and personal satisfaction. 

3. To provide employees opportunities for professional and personal development. 

4. To recognize ar.d reward employees for their personal and professional contributions to 
the institution on a regular and continuing basis. 

Personnel of Paradise Valldy Comunity College Center have developed activities designed to 
provide the participants with a means for achieving one or more of the four goals of the 
Employee Development Program. However, an assessment has never been conducted to 
determine whether employees* participation in these activities is effective in helping them to 
meet these goals, nor is there a system in place to do so. Miller (1986:425) contends that 
institutloral assessment is important to maintaining institutional quality and suggests: **A 
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determination of how well the goals of the specific area assessed are being met should be made. 
The appropriate instruments and techniques must be selected and administered to the 
constituencies Involved.** Given these realities, the following problem was studied: 
Administrators at Paradise Valiey Community College Center lacl^ a means for evaluating the 
effectiveness of the Employee Development Program. 

The Employee Development Committee at Paradise Valley Community College Center is made 
up of representatives of three of the five employee groups represented on campus (RFF- 
Residontial Faculty (four representatives). MATP-Management/Administratlve/Technical 
Personnel (five representatives), and PSA- Professional Staff (two representatives). The 
Crafts employee group and the M&O-Maintenance and Operations employ ie group are not 
represented on the oommitte^j. At a meeting of the Employee Development Committee, chaired by 
the Provost of the college, the members discussed the need to evaluate employees' participation 
in employee development activities in terms of whether their participation in the activities 
leads to the achievement of the goals of the Employee Development Program. Members of the 
Employee Development Committee reported that they had been approached by various employees 
who questioned whether their participation in certain employee development activities aided 
them in becoming more effective personally and professionally. In order to gather empirical 
data more systematically regarding this problem, the oo nmittee members decided to develop an 
extensive questionnaire that would measure whether employees found that their participation in 
employee development activities effectively enhanced their personal and professional 
development. In addition, the committee members decided to design the questionnaire in such a 
way that it could be modified each academic year for the purposes of evaluating the effectiveness 
of participation in employee development activities implemented during each academic year. 
This would provide the committee members with a means for conducting a yearly evaluation of 
participation in employee development activities. Moreover, the committee members 
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determined that they needed to identify the appropriate statistical procedures for analyzing the 
empirical data and. if necessary, a system for reporting any potential written responses. 
Lastly. thP committee members committed themselves to developing a procedure for the futura 
implementation of the questionnaire. 

In summary, as a result of this study, the Provost of Paradise Valley Community College 
Center was provided with a questtonnaire that can be revised yearly to evaluate the effectiveness 
of employees' participation in the college's employee development activities in terms of meeting 
one or more of the four goals of the Employee Development Program and to generate suggestions 
for new activities. The Provost was also given spaclfic procedures for the future administration 
and analysis of the questionnaire. 

BACKGROUND AND SlGI^RCANCE 

Several references in the literature regarding the importance of developing, implementing, 

and evaluating Employee Development Programs were reviewed. When participation in 

employee development activities aids personnel to become more effective personally and 

professionally, this participation contributes to the effective functioning of institutions of 

higher education (Roueche and Baker, 1983). Peters 0987) suggests that successful 

institutions have activities in place that aid participants in the achievement of employee 

development objectives. Peters (1987:386) writes that "we must invest in human capital as 

much as in hardware." Similarly, Craven (1986:452) quoting from a speech presented by 

C. R. Pace titled "Thoughts on Evaluation in Higher Education" writes: 

A college or university is a habitat, a society, a community, an environment, an ecosystem. 
It should be judged by the quality of life that it fosters, the opportunities for experience and 
exploration it provides, the concern for growth, for enrichment, and for culture that it 
exemplifies. The question is not just y/hax does your machine produce?" but also "how does 
your garden grow?" 
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Hekimlan (1984:2) suggests that "one of the requirements of a profession is that its members 
continue to grow in order that their clients receive the best skills and knowledge available to 
them." 

The goals of the t-mployee Development Program at Paradise Valley Community College 

Center have been documented to be the goals of weil-run institutions (Peters and Waterman, 

1982). However, the Employee Development Program has never been systematically 

evaluated. Miller (1986:425) writes: 

Institutional evaluation should use objective data where available and purposeful. ... The 
absence of objective data should stimulate those responsible for institutional evaluation to 
devise their own survey instruments, guidelines, and checklists or to use systematically 
treated Judgment as bases for decision making. 

Those who are expected to participate in institutional programs should be involved in their 

development and in their subsequent evaluation (Clagett, 1980; Naisbitt, 1982; Roueche 

and Baker, 1983). Schuster (1989:70), writing about faculty development programs, states 

that "the faculty itself must be involved in any effort to create, expand, or evaluate faculty 

development activities." Similarly, Locke (1985:3) contends that "the active participation of 

teachers in all aspects of inservice programs is seen as vital for the success of those programs." 

Peters (1987:342) suggests: 

We must involve all personnel at all levels in all functions in virtually everything: for 
example, quality improvement programs and 100 percent self-inspection; prodt'ctivity 
improvement programs; measuring and monitoring results; budget development, 
monitoring, and adjustment; layout of work areas; assessment of new technology; recruiting 
and hiring; making customer calls and participating in customer visit programs. 

Bradford and Cohen (1984:184-185) write: 

The rule of thumb is to have greater involvement when subordinates have crucial 
information or abilities and a quality solution is needed. ... The question is whether you 
want excellent performance. Groups (and individuals) can't achieve that standard if they 
don't address issues of significance and work through them together. 

Lastly, Hekimlan (1984:38) suggests that "the meaningful involvement of staff in the 

evaluation process promotes a sense of trust, awareness, and commitment." 
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In order to develop tools that will in fact generate an effective evaluation of participation in 

employee development activities, care must be taken in their development, administration, and 

analysis (Glass and Stanley, 1970; Hopkins and Glass, 1978; Isaac and Michael, 1981; 

Babble, 1975; Zemke and Rossett, 1985). Fink and Koseooff (1985:13) suggest: 

Survey and methods fall on a continuum. Some surveys can have far-reaching, 
generailzabie effects, and their methods must be scientific. Others are conducted to meet 
very specific needs; their methods may not always achieve scientific rigor, but they must 
still be valid. 

In addition, Kerlinger (1973:457) writes: 

The commonest definition of validity is epitomized by the question: Are we measuring what 
we think we are measuring? The emphasis In this question is on what is being measured. 
Although the commonest definition of validity was given above, It must be immediately 
emphasized that there is no one validity. A test or scale is valid for the scientific or 
practical purpose of its user. 

One way to address the validity of the questionnaire that was developed for this study is to 

investigate its content validity. Kerlinger (1973:458) states: 

Content validity is the rc xesentatlveness or sampling adequacy of the content-the 
subFlance, the matter, the toplcs--of a measuring Insirument. Content validation is guided 
by the question: Is the substance or content of ihis measure representative of the content or 
the universe of content of the property being measured? 

The successful achievement of the goals of the Employee Development Program is important 

in achieving the mission of Paradise Valley Community College Center. Since the outcomes of 

this study are related to involving personnel in determining the outcomes of decisions that affect 

them and involve the development of human capital, this study is related to the principles 

presented in the Politics, Law, and Economics Seminar. As noted in the study guide for the 

Politics, Law, and Economics seminar, IVIarlorana, Kelly, and Nespoli (1989:5) support the 

involvement of personnel in Employee Development Programs when they write: 

It is simply that, if the personnel engaged in the enterprise are to be meanipnfully involved, 
influential, in determining the direction and broad operating procedures of the enterprise, 
they ought to be well-informed about the philosophy, functions, and structures of the 
enterprise. 
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In addition, Martorana, Kelly, and Nespoli (1989:34) note: 

Ideally, policies to govern and guide the affairs of postsecondary education ought to emerge 
from the regularly established structures and processes of governance. In that ideal 
condition one would expect that the full range of interests within the organization would have 
a voice and a way to express ft as the policy is deveirpad and made official. 

As a result of this study, personnel of Paradise Valley Community College Center were given 

the opportunity to be better informed regarding the Employee Development Program at the 

college and were given a voice in developing the questionnaire that will eventually be used to 

evaluate the activities of the Employee Development Program. 

In summary, the purpose of this study was to produce a questionnaire designed to evaluate 

the effectiveness of employees' participation in the activities of the Employee Development 

Program in terms of helping them to achieve the goals of the Employee Development Prcs''<^m at 

Paradise Valley Community College Center. The development of this questionnaire is an 

important first step in operationalizing Strategic Planning Goal Number Three of Paradise 

Valley Community College Center: "Strengthen Paradise Valley Community College Center's 

Employee Development Program" fMaricQr^a Community CQllft^a s Strategic Plan Summaiv FY 

199Q-91 n.d.). 

PROCEDURES 

In order to design the questtonnaire and to mrAe recommendations regarding its future 
implementation and analysis, the following procedures were followed: 

1 . A member of the Employee Development Committee completed a review of related 
literature on the effective development, analysis, and administration of questionnaires. 

2. Based on the review of the related literature regarding the effective development of 
questionnaires, the Employee Development Committee members developed the goals of the 
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evaluation of the activities of the Employee Development Program. Six research hypotheses 
were developed in this regard. 

3. In order to ensure the content validity of the questionnaire to be developed, the Employee 
Development Committee members formed a suboomittee. The goal of the subcommittee was 

to identify the activities that were designed to provide participants with the opportunity to 
achieve one or more of the four employee development goals. The subcomlttee then met with the 
full committee to solicit feedback regarding their work. Lastly, the suboomittee held 
informational meetings with all full-time and part-time Maricopa County Community College 
District Governing Board approved employees at the college. The purpose of these informational 
meetings was to provide employees with the opportunity to review the list of activities that 
were identified by the Employee Development Committee as activities that were designed to meet 
one or more of the four employee development goals. At these informational meetings, 
employees were encouraged to ask questions regarding activities with which they may have been 
unfamiliar. If an employee was unable to attend one of these meetings, a member of the 
Employee Development Subcommittee met with him or her to share the list of activities. 

4. Based on the review of the related literature regarding the effective development of 
questionnaires and based on the goals of the evaluation of the activities of the Employee 
Development Program, the subcommittee developed the questionnaire. The subcommittee then 
met with the full committee to solicit feedback regarding the questionnaire. 

5. Three research experts evaluated the degree of appropriateness of the questionnaire 

in measuring whether employees perceKed their participation in the activities as effective in 
terms of the stated goal. In addition, the research experts evaluated the directions for filling out 
the questionnaire. 

6. Three research experts recommended the statistical procedures needed and a computer 
software statisttoal program that could be used to analyze the empirical data generated from the 
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future implementation of the questionnaire. Based on the review of the related iiterature 
and the recommendations of the three research experts, the subcommittee chose the statistical 
procedures and a computer software statistical program that wiil be used to analyze the 
empirical data generated from the future implementation of the questionnaire. 

7. Three research experts recommended a form that employees could eventually use 
to record their responses to the questionnaire items, in addition, the three research experts 
recommended, explained, and demonstrated a procedure for scanning into a computer data file 
the empirical data generated from the future implementation of the questionnaire. Lastly, the 
three research experts recomnriended, explained, and demonstrated a procedure for downloading 
ihe computer data file into a computer software statistical program. Based on the review of the 
related literature and the recommendations of the three research experts, the Employee 
Development Committee members chose the data form, the procedure for scanning and 
downloading the empirical data, and the computer software statistical program that will be used 
to analyze the empirical data generated from the future implementation of the questionnaire. 

3. Three research experts suggested a procedure for reporting any written responses that 
may be generated from the future implementation of the questionnaire. Based on the review of 
the related literature and the recommendations of the three research experts, the subcommltee 
developed a procedure for reporting and analyzing any written responses that may be generated 
from the future implementation of the questionnaire. 

9. Based on the review of the related literature, the Employee Development subcommittee 
members developed a procedure for the future administration of the questionnaire. The 
population to be surveyed was identified, and specific procedures for dissemination and 
collection of the completed questionnaires were developed. Three research experts reviewed the 
proposed process for administration of the questionnaire. 
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10. The queslonnaire was field tested. A raprenentative from four employee groups filled 
out the question laire and provided feedback regarding their understanding of the purpose of the 
questionnaire and the directions for filling out the questionnaire. 

Limitations and Aftfiiimptinnc: 

The assumption was made that all full-time and part-t«me Maricopa County Community 
College District Governing Board approved employees at the college were provided with an 
.opportunity to review the list of activities that were identified to meet one or more of the four 
employee development gouis. Generalizations from the data generated from the future 
implementation of the questionnaire could be limited if some employees who eventually 
participate in the survey are unfamiliar with some of the activities of the Employee 
Development Prog«'am. 

Definiiinn of Tarmft 

1 . Employee development program: A program designed by an institution to foster the 
personal and professional success of its employees. 

2. Program evaluation: The process of specifying, defining, collecting, analyzing, and 
interpreting information about designated aspects of a given program and using that information 
to arrive at value judgmerts among decision alternatives regarding tne installation, 
continuation, modification, or termination of a program" (Craven, 1986:434). 

3. Questionnaire: A printed question and answer fonnat such as surveys, polls, rnd 
checklists to which individuals respond by choosing from lists of prepared answers or writing 
in original responses (Zemke and Rossett, 1985). 

4. Survey: "A method of collecting information directly from people about their feelings, 
motivations, plans, beliefs, and personal, educational, and financial background. It usually 
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takes the form of a questionnaire that someone fills out alone or with assistance, or it can be 
conducted as an interview in person or on the telephone" (Fink and Kosecoff, 1985:13). 

RESULTS 

A literature review on the effective development, analysis, and administration of 

questtonnaires was completed. Fink and Kosecoff (1985:15) write that questionnaires **can be 

used to make policy or plan and evaluate programs and conduct research when the information 

you need should come directly from people.** In addition, Isaac and Michael (1981:130) write 

that the advantages of using quvStionnaires in educational research are that they are 

*'inexpensive; wide-ranging; can be well designed, simple and clear; are self-administering; 

and can be made anonymous.** Moreover, Fink and Kosecoff (198520) conclude that \a\id and 

reliable informatton is obtained by using the most rigorous methods available. This means 

having knowledge of survey design and sampling procedures and questionnaire construction.** 

Lastly, Lenning (1986:281) writes: 

It should be apparent by now that there are primary criteria for indicator, measure, and 
data collection method selection other than just reliability and valkJity, including 
appropriateness to the analytical procedures and tests planned, ease of data collection, case 
of scoring and tabulation, collectton or analysis cost, and whether program planners and 
administrators will be able to understand readily the implications of such data. 

According to Fink and Kosecoff (1985), when developing questionnaires, the following 
questions need to be considered: What are the infonnation needs or hypotheses? What types of 
items will be included? How will the items be structured? Do the items have content validity? 
What level of measurement will be used? What type of measurement scale will be used? How 
will the questionaire be formatted? Who will be surveyed? 

When discussing guidelines for institutional evaluation. Miller (1986:425) writes: 
**A clear aefinition of the goals or the assessment, as distinct from the goals of the specific area 
being assessed, should be made. The main focus of assessment should be on evaluatton of 

Er|c 14 
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educational quality as measured by goal-oriented outcomes.** When discussir^n the development 

of the content of a questionnaire, Fink and Kosecoff (1985:23) stata: '*Deciding on a survey's 

content means setting the survey's boundaries so that you can write the correct questions." 

Lenning (1986:282) suggests that when deciding on the goals of the assessment of a program, 

an initial wholistic and *broad-band* focus followed by a more in-depth and detailed 
*narrow-band* focus [can be effective]. In the broad-band phase, secondary data that are 
already available and questionnaire or interview data that have a gross level of focus on 
broad areas in terms of needs (plus broad, open-ended questions regarding needs) are 
examined to identify important need areas that call for a more detailed focus. Then detailed 
and in-depth needs data that can support need understanding and setting need priorities 
(including open-ended data ai that level of detail) are collected for those need areas that 
seemed most important in the broad-band phase. 

Isaac and Michael (1981:133) note that for a qi'astionnaire to be effective, "it rer jires a 

careful, clear statement of the problem underlying the questionnaire." Moreover, Kerlinger 

(1973:485) writes that "the purpose of each question is to elicit information that can be used 

to test the hypotheses of the research." Barton (1987:15-16) writes: 

The hypothesis is a statement of the outcome which you anticipate in this investigation. Tnis 
is a statement about the relationships between variables. It is always a simple statement of 
what you expect to be shown, and should not involve such words as "should" or "ought", and 
it will not be stated in terms of morals or ethics. As a final point, in any project there can 
be more than one hypothesis. In any particular project you should use only the number of 
hypotheses that you can investigate in the time available and with the resonrces at hand. 

When considering the types of items to be included in a questionnaire. Babbie (1975:106) 
writes: 

The term "questionnaire" suggests a collection of questions, but an examination of a typical 
questionnaire will probably reveal as many statements as questions. This is not without 
reason. Often, the researcher is interested in determining the extent !o which respondents 
hold a particular attitude or perspective. 

In addition. Babbie (1975) advises that questbnnaires may contain open-ended and or closed- 
ended question s. Open-ended questions are questions in >Mlch "the respondent is asked to 
provide his own answer to the question. Ir^ the other case, closed-ended questions, the 
respondent is asked to select his answer from among a list provided by the researcher" (Babbie 
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(1975:107). Kerllnger (1973:485) writes: 

Some information can best be obtained with the open-ended question-reasons for behavior, 
intenttons, and attitudes. Certain other information, on the other hand, can be more 
expeditiously obtained with closed questions. If all that is required of a respondent is his 
preferred choice of two or more alternatives, and these alternatives can be clearly 
specified. It would be wasteful to use an open-ended questton. 

Similarly, Isaac and Michael (1981:133) suggest that ''objectivity is important. Lengthy 

subjective, open-ended answers are difficult for the respondent to write and for the 

investigator to evaluate. If the possible categories of responses can be anticipated, these should 

be offered as alternatives to an objective question." When considering questionnaire length. 

Fink and Kosecoff (1965:42) conclude: 

The length of a survey form depends upon what you need to know and how many questtons are 
necessary so that the resulting answers will be credible. Self-administered questionnaires 

... are generally limited to thirty minutes and contain the fewest Items Another 

consideration is the respondents. How much time do they have available, and will they pay 
attentton to the survey? 

When discussing how the closed-endeci items of a questionnaire should be structured. 
Babbie (1975:107) writes: "The response categories provided should be exhaustive: they 

should include all the possible responses that might be expected Second, the answer 

categories must be mutually exclusive: the respondent should not be compelled to select more 
than one." In addition. Babbie (1975:108) suggests that items should be clear: "Questionnaire 
items should be ptacise so that the respondent knows exactly what the researcher wants an 
answer to." Moreover, Kerlinger (1973:485-487) s'-3gests seven criteria for writing 
questionnaire Items: 

1. Is the questton related to the research problem and the research objectives? 2. Is the 
typecf questton ti^ It and appropriate? 3. is the item clear and unambiguous? 4. Is the 
questton a lead.ng questton? 5. Does the question demand knowledge and informatton that the 
respondent does not ha/e? 6. Does the question demand personal or delicate material that 
the respondent may resist? 7. Is the questton loaded with social desirability? 

Developers of questtornaires must concern themselves with the content validity of the items 

included in the questionnaire. Fink and Kosecoff (1985:50) propose: 
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A survey can be validated by proving that its items or questions accurately represent the 
characteristics or attitudes that they are intended to measure. Content validity is usually 
established by asking experts whether the items are representative samples of the attitudes 
and traits you want to survey. 



In addition, Kerlfnger (1973:459) writes: 



Content validation, then, is basically judgmental. The items of a test must be studied, each 
Item being weighed for its presumed representativeness of the universe. This means that 
each item must be judged for its presumed relevance to the property being measured, which 
is no easy tasl^. Usually other "competent" judges should judge the content of the items. The 
universe of content must, if possible, be clearly defined; that is the judges must be 
furnished with specific directions for making judgments, as well as with specification of 
what they are judging. 

When developing items for a questionnaire, the type of measurement to be used must be 

chosen. Kerlinger (1973:427-435) defines measurement as 

"the assignment of numerals to objects or events according to rules." A rule is a guide, a 
method, a command that tells us what to do. A mathematical rule is t, a function; f is a rule 
for assigning the objects of one set to the objects of another set. In measurement a rule 
might say: "Assign the numeral 1 through 5 to individuals according to how nice they are. 
If an Individual is very, very nice, let the number 5 be assigned to him. If an individual is 
not at all nice, let the number 1 be assigned. Assign to individuals between these limits 
numbei^ between the limits." The rules to assign numer als to objects define the kind of 
scale and the level of measurement. 



Similarly, Fink and Kosecoff (1985:33) suggest that "with rating, the respondent places the 
item being rated at some point atong a conJnuum or in any one of an ordered series of 
categories; A numerical value !s assigned to the point or category." Kerlinger (1973) discusses 
four levels of measurement: nominal, ordinal, interval, and ratio. Kerlinger (1973:435- 
438) defines these levels of measurement as follows: 



The lowest level of measuremint is nominal measurement The r jmbers assigned to 

objects are numerteal without iiaving a number meaning; they cannot be ordered or added. 

If individuals or groups are assigned 1, 2, 3 these numerals are merely names. 

Ordinal measurement requires . . . that the objects of a set can be rank-ordered on an 
operattonally defined characteristic or property. Ordinal numbers indicate rank order 
and no:;iing more. The numbers do not indicate absolute quantities, nor do they indicate that 
the intervals between the numbers are equal. Interval or equal-interval scales possess 
the characteristics of nominal and ordinal scales, especially the rank-order characteristic. 
In addition, numerically equal distances on interval scales represent equal distances In the 
property being measured. A ratio scale, in addition to possessing the characteristics of 
nominal, ordinal, and interval scales, has an absolute or natural zero that has empirical 
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meaning. If a measurement is zero on a ratio scale, then there is a basis for saying that 
some otsject has none of the property being measured. 

In addition to deciding on the appropriate level of measurement, researchers must decide 

on the measure scale to be used. Attitude scales have been used to evaluate educational programs 

(Kerlinger, 1973). Kerlinger (1973:495-496) defines an attitude as 

an organized predisposition to thinl<, feel, perceive, and behave toward a referent or 
cognitive object. It is an enduring structure of beliefs that predisposes the individual to 
behave selectively toward attitude referents. A referent is a category, class, or set of 
phenomena: physical objects, events, behaviors, even constructs. People have attitudes 
toward many different things: ethnic groups, institutions, religion, educational issues and 
practices, the Supreme Court, civil rights, private property, and so on. 

Moreover, Anastasl (1972:479-480) defines an attitude as 

a tendency to react favorably or unfavorably toward a designated class of stimuli, such as a 
national or racial group, a custom, or an institution. It is evident that, when so defined, 
attitudes cannot be directly observed, but must be Inferred from overt behavior, both 
verbal and nonverbal. In more objective terms, the concept of attitude may be said to 
connote response consistency with regard to certain categories of stimuli. 

When discussing attitude scales, Isaac and Michael (1981:142) define a scale as 

a measuring device allowing the assignment of symbols or numbers to individuals, or their 
behaviors, by rule. Such an assignment indicates the individuaFs possession of a 
corresponding amount of whatever the scale Is claimed to measure. 

Regarding attitude scales, Anastasl (1972:480) observes: 

Attitude SO' s . . . typically yield a total score indicating the direction or intensity of the 
Indlvkjual's attitude toward a company, group of people, policy, or other stimulus category. 
In the construction of an attitude scale, the different questions are designed to measure a 
single attitude or unidimensional variable, and some objective procedures are usually 
followed in the effort to approach this goal. Ar^ employee attitude scale, for example, yields 
a single score showing the individual's degree of job satisfactton or over-all attitude toward 
the company. 

Babbie (1975:351) indicates that a Likert scale developed by Rensis Likert should be used 
when identical response categories will be used for "several items intended to measure a given 
variable (and when] each item (must) be scored in a uniform matter." Isaac and Michael 
(1981:142) define Likert-type or summated rating scales as containing 
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a set of items, all of which are considered approximately equal in attitude or value loading. 
The subject responds with varying degrees of intensity on a scale ranging between extremes 
such as agree-disagree, lil(e-dislil<e or accept-reject. The scores of the position responses 
for each of the separate scales are summed, or summed and averaged, to yield an individual's 
attitude score. Summated rating scales seem to be the most useful in behavioral research. 
The main advantage of a summated scale lies in the greater variance obtained. The 
disadvantage, as with all scales, is the vulnerability of this variance to biasing response sets 
(e.g., the over-rater or the under-rater). 

Babbie (1975:351) concludes that **the Ulcert method is based on the assumption that the 

overall score based on responses to the many items seeming iO reflect the variable under 

consideration provides a reasonably good measure of the variable.** Moreover, nominal scales 

are used when the researcher is interested in data regarding a particular group to which a 

respondent belongs {Pink and Kosecoff, 1985). Lastly, ordinal scales are used when the 

investigator is interested in data regarding how respondents ranl^ ordered items {Fmk and 

Kosecoff, 1985). 

When considering the format of a questionnaire, Finl^ and Kosecoff (1985) suggest that 

decisions need to be made regarding the ordering of questions and the aesthetics of the 

questionnaire. Similarly, Babbie (1975:111) warns that the 

format of a questionnaire can be Just as important as the nature and wording of the questions 
asl<ed. An improperly laid out questtonnaire can lead respondents to miss questions, can 
confuse them as to the nature of the data desired, and, in the extreme, can lead to respondents 
throwing the questionnaire away. 

fink and Kosecoff (1985:43-44) provide the following suggestions for formatting 
questionnaires: 

All surveys should be preceded by an introduction, and the first set of questions should be 
related to the topic described in it. People sometimes respond best when the first questions 
ask for objective facts. Once they become used to the survey and more certain of its 
purposes, they will usually provide the answers to relatively subjective questions. 
Questions should proceed from the most familiar to the least. Place relatively easy-to- 
answer questions at the end. When questionnaires are long or difficult, respondents may get 
tired and answer the last questions carelessly or not answer them at all. Avoid many items 
that look alike. Twenty items, all of which ask the respondent to agree or disagree with 
statements, may lead to fatigue or boredom, and the respondent may give up. To minimize 
loss of interest, group questions and provkle transittans that describe the format or topic. 
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Questions that are relatively sensitive should be placed toward the end. Finally, questions 
should appear in logical order. 

In terms of aesthetical considerations when developing a questionnaire, Fink and Kosecoff 
(1985:45) note: "A questionnaire's appearance is vitally important. A self-administered 
questionnaire that is hard to read can confuse or irritate respondents. The result is loss of 
data." Similarly, Babbie (1975:111) advises that "the questionnaire should be spread out and 
uncluttered. The researcher should maximize the 'white space' in his instrument." Moreover, 
when malting decisions regarding response format. Fink and Kosecoff (1985) suggest that if the 
researchers choose to ask about a topic that they know in advance will not be relevant to 
everyone in the survey, the questionnaire must be designed in such a way that it allows for 
respondents to skip those items. Babbie (1975:122) urges that "the format should take 
intended processing methods into account. If the questionnaire is to be read by an optical- 
sensing machine, then the researcher must check his format against the requirements of the 
machine." lasWy, Babbie (1975:114) writes: "All the foregoing discussion [regarding format 
considerations for developing questionnaires] should point out the way in which seemingly 
theoretical issues of validity and reliability are involved in so mundane a matter as how to put 
questions on a piece of paper." 

When considering who should be surveyed, researchers turn their considerations to sample 
size. Isaac and Michael (1981:132) write: "Whenever practical, especially if a survey 
touches on controversial matters or will lead to an important decision or conclusion, it is well 
to include all possible respondents." Similarly, Miller (1986:425) advises: "All persons vA\o 
are affected and interested in the programs under review should be continually made aware of 
and often involved in the assessment process." In addition, Kerlinger (1973:127-128) notes: 

Use as large samples as possible. Whenever a mean, a percentage, or other statistic is 
calculated from a sample, a population value is being estimated. I^rge samples are not 
advocated because large numbers are good in and of themselves. They are advocated in order 
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to give the principle of randomization, or simply randomness, a chance to "work," to speak 
somewhat anthropomorphically. 

In the process of developing questionnaires, researchers must make decisions regarding the 

analysis of the empirical data that will be generated from the future administration of the 

questionnaires. Craven (1986:446) writes: 

What analytical methodologies are needed? (Researchers must] specify those analytical 
methodologies needed to convert the data elements into the required evaluation infonnation. 
The methodologies may vary in complexity, depending upon the specified Information 
outputs. In this regard, there is nothing Inherently superior about evaluations that employ 
sophisticated analytical techniques; the important criterion is whether the analytical 
methodok)gies and the resulting Information meet the requirements of the evaluation 
questions and objectives in a valid, reliable, and objective manner. Further, as a basic 
guideline, it is preferable to keep the analytical requirements within a moderate range of 
complexity in order to facilitate participant understanding and communication of the 
evaluation findings. 

In addition. Fink and Kosecoff (1985:73) write: 

Analyzing data from surveys means tallying and averaging responses, looking at their 
relationships, and comparing them-sometimes over time. The appropriate analysis 
method to use Is dependent upon the answers to at least five questtons: (1) How many 
people are you surveying? (2) Are you looking for relationships or associations? (3) 
Will you be comparing groups? (4) Will your survey be conducted once or several times? 
(5) Are the data recorded as numbers and percentages or scores and averages? 

When discussing methods typically used to analyze data generated from questionnaires. Fink and 
Kosecoff (1985:73) write: 

Commonly used survey data analysis techniques include the following: (1) descriptive 
statistics (mean, mode, meJian, numbers, percentage, range, standard deviations) (2) 
correlations (Spearman rank-order, Pearson product-moment) (3) comparisons (Mann- 
Whitney U, chi-square, t-test, analysis of variance) and (4) trends (repeated measures 
analysis of variance, McNemar test). 

If Investigators are interested in testing for significant differences among the means of two 
or more groups, "the statlstteal technique known as the analysis of variance (ANOVA) Is used to 
determine whether the differences among two or more means are greater than would be 
expected by chance alone. ANOVA employs the Rest, which is the ratio of two Independent 
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variance estimates" (Hopkins and Glass. 1978:032-336). Isaac and Michael (1981:182) 
write: 

Analysis of variance . . . answers the question, Is the variability between groups large 
enough in comparison with the variability within groups to Justify the inference that the 
means of the populations from which the different groups were sampled are not all the 
same? In other words, if the variability between groups means is large enough, we can 
conclude they probably come from different populations and that there is a statistically 
significant difference present in the data. 

Moreover, Isaac and Michael (1981:182) conclude: 

While the analysis of variance is the first step in the analysis of these more complex 
designs, it is only a preliminary and exploratory tool. If a significant F-ratio is obtained, 
the researcher only knows that somewhere in his data something other than chance Is 
probably opera;ir:g. He next must attempt to isolate the presence, nature, and extent of this 
non-chance Influence. 

Kerilnger (1973:236) concurs when he writes: 

In analysis of variance, an overall Ftest, If significant, simply Indicates that there are 
significant differences somewhere in the data. Inspection of the means can tell one, though 
imprecisely, whteh differences are important. To test hypotheses, however, more or less 
controlled and precise statistical tests are needed. 

In order to determine where the significant differences are, researchers must use a post hoc 

test. Kerilnger (1973:235) advocates the use of the Scheffe test: 

The Scheffe test, if used with discretion. Is a general method that can be applied to all 
comparisons of means after an analysis of variance. If and only If the Ftest Is significant, 
one can test all the differences between the means; one can test the combined mean of two or 
more groups against the mean of one other group; or one can select any combination of means 
against any other combination. 

If Investigators, using ranked data, are interested in testing for significant differences among 
two or more groups, a one-way analysis of variance (Kruskal-Wallis Test) is employed. 
Kerilnger (1973:290) writes: "There are . . . research situations in which the only form of 
measurement possible Is rank order, or ordinal measurement. The Kruskal and Wallis test Is 
most useful in such situations." 

If a queslionaire contains open-ended questions, a method for analyzing the responses must 
be chosen. "Content analysis is an objective and quantitative method for assigning types of 
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verbal and other data to categories" {Kerlinger.1 973:41 7). Kerllnger (1973:528) suggests 

that the first step in conducting a content analysis is "to define U, the universe of content that is 

to be analyzed. Categorization, or the partitioning of U, is perhaps the most important part of 

content analysis, because it is a direct r( lection of the theory and the problem of a study. It 

spells out, in effect, the variables of the hypotheses." In addition, Kerlinger (1973:528) notes 

that when conducting a content analysis, the units of analysis must be identified: 

The word is the smallest unit. (There can ever, be smaller units: letters, phonemes, etc.) 
It is also an easy unit to work with, especially in computer content analysis. The theme is 
a useful though more difficult unit. A theme is often a sentence, a proposliion about 
something. Themes are combined into sets of themes. It should be emphasized, . . . that if the 
themes are complex, content analysis using the theme as the unit of analysis is difficult and 
perhaps unreliable. 

Futhernrore, Kerlinger (1973:530), when discussing the assignment of numbers to the objects 
of a content analysis, suggests: 

There are three or more ways to assign numbers to the objects of the content analysis U. 
The first and most common of these corresponds to nominal measurement: count the number 
of objects in each category after assigning each object to its proper category. A second form 
of quantification is ranlcing, or ordinal measurement. If one is worthing with not too many 
objects to be ranlced--say not more than 30--judges can be asl(ed to ranic them according to 
a specified criterion. A third form of quantification is rating. Children's compositions, for 
example, can be rated as wholes for degrees of creativity, originality, inner-direction and 
other-direction, achievement orientation, interests, values, and other variables. 

According to Kerlinger (1973:530) certain conditions need to be met before quantification is 

worthwhile or justified: 

(1) to ount carefully (or otherwise quantify) when the materials to be analyzed are 
representative, and (2) to count carefully when the category items appear in the materials 
in sufficient numbers to justify counting (or otherwise quantifying). The reason for both 
conditions is obvious: if the materials are not representative or if the category items are 
relatively infrequent, generalization from statistics calculated from them is unwarranted. 

Finally, when discussing the analysis of data resulting from both closed-ended and open-ended 

questionnaire items, Lenning (1986:283) writes: 

The analyses used should be understandable and meaningful to those who will use the 
information coming out of the analyses. Also, in doing the analyses, care must be talcen that 
different scales are not erroneously equated and that hard and soft data are integrated in a 
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way that does not mislead. For example, changing soft data to nur.ibers can misrepresent the 
original picture if one is not careful. Subjective analysis ("eyeballing") and logic are often 
called for in comparing data. If materials cannot meet the criteria, they can be u^ed only for 
heuristic and suggestive purposes and not for relating variables to each other. 

In addition to choosing the data analysis methods, a decision needs to be made regarding the 
use of a computer to analyze empirical data generated from the questionnaires (Isaac and 
Michael, 1981; Craven, 1986). Questions to ask and answer regarding the use of a computer 
are: How will the information be inputted into the computer? Does the institution have the 
hardware and software necessary to scan the data into a data file? If so, wiH a special answer 
sheet be needed? If a special answer sheet is needed, will the respondents be required to use a 
special marking instrument? What computer hardware and software are available to analyze 
the data? Does the institution have expert personnel in data analysis using computer hardware 
and software? If not, will there be sufficient budget money to hire a data analysis expert? 

After a questionnaire has been de. sloped and decisions have been made regarding the analysis 

of the data, the questionnaire snould be pilot tested. Fink and Kosecoff (1985:42) advise: 

When pilot testing, anticipate the actual circumstances in whteh the survey will be 
conducted and make plans to handle them. Choose respondents similar to the ones who will 
eventually complete the survey, and enlist as many people as you can. For reliability, 
focus on the clarity of the questions and the general format of the survey. Pilot testing 
also bolsters validity because it can help you see that all topics are included and that 
sufficient variety in the responses is available-if people truly differ, your survey will 
pick up those differences. 

Zemke and Rossett (1985:6) offer similar advice: "Always pilot the questionnaire. Ask a 
sample group or at least two individuals to comment on clarity and format. This feedback 
indicates whteh questions and instructions should be reworded or edited, reducing the 
possibility of misinterpretation." Lastly, Isaac and Michael (1981:136) offer the following 
additional suggestton: "Analyse [sic] the results to assess the effectiveness of the trial 
questionnaire to yield the information desired." 
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Moreover, researchers need to develop a plan for an on-going assessment of institutional 

programs. Miller (1989:425) advises that "the process of collecting data should be established 

in such a way that it can continue beyond the first self-assessment as a routine function of the 

master planning ^nd decision-mal<ing process." Futhermore, Miller (1989:425) suggests that 

"essential to effective self-assessment is the periodic evaluation of the system itself. The 

system should be cost-effective in both dollars and human time spen! to provide vital 

information for decision mal<ing." Lastly, Miller (i 986:426) advises: 

A plan for evaluating the evaluation should be included. Most evaluattan reports make little 
or no provision for evaluation of their effectiveness. Such evaluations of evaluations are 
useful as testimony to the importance of evaluation in future improvement, for providing 
systematic checl<points on progress, and for providing a procedure for orderly modifications 
based upon subsequent findings. 

Lastly, when the quetilonnaire has been developed, researchers must concern themselves 

with the issues related to the administration nf the questionnaire In order to ensure an 

acceptable rate of return. Finl< and Kosecoff (1985:45) write: "Self-administered 

questionnaires require much preparation and monitoring to get a reasonable response rate. 

These questionnaires are given directly to people for completton and very little assistance is 

available in case a respondent does not understand a question." Rnl< and Kosecoff (1985:46) 

offer the following suggestions for using self-administered questionnaires: 

1. Send respondents a preletter telling them the purpose of your survey. This should warn 
people that the survey is coming, explain why the respondents should answer the questions, 
and tell them about who is being surveyed. 2. Prepare a short, formal letter to accompany 
the questionnaire form. If you have already sent a preletter, this one should be very 
concise, it should again describe the survey and questionnaire aims and participants. 3. 
Offer to send respondents a summary of the findings so they can see just how the data are 
used. (If you promise this, budget for it) 4. If you asl< questions that may be construed as 
personal-such as sex, age, or income-explain why they are necessary. 5. Keep 
questionnaire procedures simple. Provide stamped self-addressed envelopes. Keep page 
folding to a minimum so respondents do not feel they are involved in complicated physical 
ojtivities. 6. Keep questtonnaires as short as you can. Asl< only the questions you are sure 
you need and do not crowd them together. Give respondents enough room to write and be sure 
each question is set apart from the next. 7. Consider incentivp^. This may encourage people 
to respond. These may range from money and stamps to pens or fbod. 8. Be prepared to 
follow up or send reminders. These should be brief and to the point, it often helps to send 
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another copy of the questtonnaire. Do not forget to budget money and time for these 
additional mailings. 

Similarly, Nowacli (19S0) advises tfiat a cover letter addressed pe sonally to each employee 

and signed by the chief operating officer of the institution snould accompany the questionnaire. 

In addition, Isaac and Michael (1981:136) advise that the cover letter to a questionnaire should 

"establish a reasonable, but firm, return date." Nowack (1990:84) advises that the cover 

letter should include "how and to whom respondents should return the questfonnalres." 

Moreover, Nowack (1990:84) suggests that employees should be advised that they will be 

receiving a questionnaire: 

Alert Managers and employees ahead of time that a questtonraire is being developed and will 
be sent out. Make special presentations to managers alerting them to the purpose of the 
questionnaire. Use organizational communication channels such as company newsletters or 
announcements to describe the importance of the questtonnaire and the anticipated use of 
the results-betore it is mailed out. Encourage supervisors and managers to make 
announcements encouraging employees to fill out questionnaires and to return completed 
ones during staff and team meetings. 

Babbie (1975:260) offers the following advice regarding effective procedures for using 
self-administered questtonnaires: 

Scire recent experimentation has been conducted with regard to the home delivery of 
questtonnaires. A research worker delivers the questionnaire to the home of sample 
respondents and explains the study. Then, the questtonnaire is left for the respondent to 
complete, and it is pteked up subsequently by the researcher. On the whole, the appearance 
of a research worker, either delivering the questionnaire, picking it up, or both, seems to 
produce a higher completion rate than is normally true for straightforward mail surveys. 

Lastly, when devising procedures for using self-administered questionnaires, researchers 

should preserve the anonymity of the respondents. Fink and Kosecoff (1985:42) suggest that 

"the use of sun/eys and concern for ethical issues are completely interwoven. Surveys are 

conducted because of the need to know; ethical consklerattons proicwi the individual's right to 

privacy or even anonymity." Isaac and Mtehael (1981:135) write: "In order to encourage 

honest and frpnk answers, some survtys are designed to be returned anonymously. This is .T>ore 

likely to occur whore the survey is getting at highly personal or controversial informatton." 
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Moreover, Nowack (1990:84) writes: "Make sure that participatton is voluntary and either 
anonymous or oonfldential. Emptoyees who feel coerced into participating may comply, but may 
provide incomplete or biased answers.** 

Based on the review of the related literature regarding the effective development of 
questionnaires, the Employee Developee Devetopment Committee members developed the goals of 
the evaluatton of the activities of the Employee Dovelopment Program. The goals are as follows: 

1 . To reanquaint ali full-time and part-time Maricopa County Community College District 
Governing Board approved employees at the college with the four goals of the Employee 
Development Program. 

2. To familiarize all full-time and part-time Maricopa County Community College District 
Governing Board approved employees at the coiiege with the opportejnities that the Employee 
Development Program provides them in the way of activities. 

3. To emphasize that participation in employee development activities is voluntary. 

4. To inform all full-time and part-time Maricopa County Community College District 
Governing Board approved employees at the college of the employee development goal or 
goals that each of the employee development activities were designed to help them meet. 

5 . To develop a questionnaire designed to: 
a Determine if employees perceive that their participation in employee development 
activities was effective in aiding them to achieve the employee development goal or goals 
tor which the activities were designed. 

b. Determine if the m^ ;nbers of the four majt ,nploy«e groups (RFP~Resldentii 
Faculty, MATP-ManagemenVAdministrativer.bChnical Personnel, M&O-Maintenance 
and Operations, : :id PSA~Professional Staff, ) significantly differ regarding their 
assessments of the activities of the Employee Development Program. (Since the college 
employs only one person who belongs to the "Crafts" employee group, the Crafts 
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employee group will be grouped with the M&O-Maintenance and Operations employee 
group). 

a Determine which of the employee development goals rieeds more attention in terms 
of developing activities to help employees meet the goal. 

d Determine if the members of the four major employee groups (RFP-Residential 
Faculty, MATP-Management/Administrative/Technical Personnel, M&O-Maintenance 
and Operations, and PSA-Professional Staff, ) significantly differ regarding their 
assessments of which of the employee development goals needs more atlention in terms of 
developing activities to help employees meet the goal. (Since the college employs only 
one person who belo*^gs to the Xrafts** employee group, (he Crafts employee group will 
be grouped with the M&0»Maintenance and Operations employee group). 

e. Determine the percentage of employee participation in each of the employee 
development activities. 

f . Gain suggestions for new employee development activities that would potentially 
result in the personal and professional development of the participants. Based on this 
assessment, the Employee Development Committee members will be provided with data 
that can help them to make decisions regarding the additbn of new employee development 
activities. 

g Elicit feedback regarding the design and the administration of the questionnaire. 
In order to ensure the content validity of the items to be included in the questionnaire, the 
subcommittee of the Employment Development Committee identified the employee development 
activitiec that were designed to provide participants with the opportunity to achieve one or 
more of the four employee development goals. Some activities were listed under two goals 
because the committee determined that those activities were designed to help participants meet 
two of the emptoyee development goals. The subcommittee members presented their findings to 
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ihe full committee. The full committee revised the list of activities. This final list was used to 
develop the questionnaire (Appendix A). 

Lastly, in order to ensure the content validity of the questionnaire, informational meetings, 
led by members of the Employee Development Subcommittee, were held with all employee 
groups in order to provide all full-time and part*time Maricopa County Community Coliege 
District Governing Board approved employees at the college with an opportunity to review the 
list of activities. At these meeting, employees asked questtons regarding activities with which 
they were untamiliar. Employee Development Subcommittee members emphasized that 
participation in employee development activities was voluntary. As a result of these meetings, 
employees at the college became reacquainted with the four goals of the Employee Development 
Program and became more familiar with the voluntary opportunities that the Enfiployee 
Development Program provides them in the way of activities. Moreover, employees were 
informed regarding the emptoyee development goal or goals that the activities ware designed to 
help them meet. In addition, employees were told that they would be eventually invited to 
complete anonymously a questionnaire designed to evaluate how effectively their participation 
in the activities helped them to achieve one or more of the employee development goals. Lastly, 
members of the Employee Development Subcommittee met individually with employees who 
were unable to attend an Informational meeting. The agenda used for the informational meetings 
was used for the individual meetings with employees. 

Based on the goals of the evaluation, a review of the related literature, and advice from three 
research experts, the Employee Development Subcommittee devebped the questionnaire. The 
full commitee reviewed the questionnaire and approved its future use (Appendix B). The 
questionnaire was designed to evaluate the effectiveness of employee participation in employee 
development activities In terms of the employee development goals of the college. Activities that 
were designed for the participation of only one employee group were not included in the 
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questiORnaire. In addition, the questionnaire was designed to generate data that would allow for 
an analysis of differences among the four major employee groups at the college In terms of 
perceived e^octiveness of the activities of the Employee Development Program. Five research 
hypotheses were developed to address this concern. The five research hypotheses are stated as 
follows: 

1 . Hypothesis One: There are significant differences In the overall mean effectiveness 
ratings of the actW>!es of the Employee Development Program among the four major 
employee groups at Paradise Valley Community College Center. 

2. Hypothesis Two: There are significant differences In the overall mean effectiveness 
ratings of the activities designed to meet goal number one of the Employee Development Program 
among the four major employee groups at Paradise Valley Community College Center. 

3. Hypothesis Three: There are significant differences In the overall mean effectiveness 
ratings of the activities designed to meet goal number two of the Employee Development Program 
among the four major employee groups at Paradise Valley Community College Center. 

4. Hypothesis Four: There are significant differences In the overall mean effectiveness 
ratings o! the activities designed to meet goal number three of the Employee Development 
Program among the four major employee groups at Paradise Valley Community College Center. 

5. Hypothesis Five: There are significant differences In the overall mean effectiveness 
ratings of the activities designed to meet goal number four of the Employee Development 
Program among the four major employee groups at Paradise Valley Community College Center. 

The questionnaire was also designed to determine which of the employee development 
goals needs more attention in te' .is of devetoping activities to help employees meet the goal. 
Research hypothesis number six was developed to address this concern. Research hypothesis 
number six Is stated as follows: There are significant differences In the assessments of which of 
the employee development goals needs more attention In temis of developing activities to help 
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employees meet the goal among the foiir major employee groups at Paradise Valley Community 
College Center. 

The questionnaire was also designed to identify the following three factors: 

1 . The percentage of employee participation in each of the employee development activities 
included in the questionnaire. 

2. New activities that would potentially result in the personal and professional development 
of the participants. 

3. Suggestions for improving the design rnd the administration of the questionnaire. 
Finally, the questionnaire was designed so that it could be edited and then used each academic 
year to evaluate wholistically the effectiveness of the activities of the Employment Development 
Program. 

Based on the goals of the evaluation, a review of the related literature, and advice from three 
research experts, decisions were made regarding the statistical procedures to be used to analyze 
the empirical data generated from the future implementation of the questionnaire. The following 
descriptive statistics will be used to analyze the data generated from items two through 
forty-four: 

1 . Number and rate of return of the questionnaires for all employee groups. 

2. Number and rate of return of the questionnaires per emptoyee group. 

3. Mean, mode, standard deviation, and variance for each activity for all employee groups. 

4. Mean, mode, standard deviation, and variance for each activity per emptoyee group. 

5. Overall mean, standard deviation, and variance for all activities for all employee groups. 

6. Overall mean, standard deviation, and variance for all activities per employee group. 

7. Mean, standard deviation, and variance for activities designated under each of the four 
goals for all employee groups. 
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8. Mean, standard deviation, and variance for activities designated under each of the four 
goals per employee group. 

9. Number of respondents and percentage of respondents rating each activity for aM 
employee groups. 

1 0. Number of respondents and percentage of respondents rating each activity per employee 
group. 

The following descriptive statistics will be used to analyze the data generated from item 
forty-five: 

1 . Number of responses for each of the four goals for all employee groups. 

2. Number of responses for each of the four goals per employee group. 

3. Percentage of responses for each of the four goals for all employee groups. 

4. Percentage of responses for each of the four goals per employee group. 
In order to test the five hypotheses regarding significant differences in the effectiveness 

ratings of the activities of the Employee Development Program among the four major employee 
groups at Paradise Valley Community College Center, five one-way ANOVA tests will be 
employed. If the Ftests are significant, the Scheffe post hoc test will be employed to discern 
where the dnferences are. Lastly, in order to test tor significant differences in the assessments 
of which of the employee development goals needs more attention in terms of developing 
activities to help employees meet the goai among the four major employee groups at Paradise 
Valley Community College Center, the Kruskal Wallis one-way ANOVA by Ranks test will be 
employed. 

The three research experts consulted for this study recommended the General Ptirpoae Data 
Sheat I form number 19543 ^General Purpose Hflta Shflet I. 1990) (Appendix C) as the form 
that employees will eventually use to record their responses to the questionnaire items. In 
addition, the three research experts explained and demonstrated the procedures for scanning 

32 



29 



empirical data into an ASCII computer data tile using the National Computer Systems Sentry 
3000 optical scanning machine owned by Paradise Valley Community College Center. The three 
research experts recommended that when the completed questionnaires are eventually received, 
the researchers should number them. This procedure will allow the researchers to spot check 
the data entry into the ASCII computer data file for accuracy . In addition, the three research 
experts recommerxJed the use of an IBM-PC and the use of the ABstat version 6.02 computer 
software statistical program (ABsiat» 1989) for analyzing the data generated from the future 
administration of the questionnaire. Lastly, the three research experts explained and 
demonstrated how to dowr.iuad the ASCII computer data file into an ABstat version 6.02 computer 
software siat^stioal proQram (ABslat 1989). The Employee Development Committee members 
accepted the recommendations of the three research experts and decided to hire a research 
expert to perform the statistical analyses on the empirical data generated from the future 
administration of the questionnaire. 

Based on the review of the related literature and the recommendations of three research 
experts, the subcommitee developed the following procedure for reporting and analyzing any 
written responses that may be generated from questionnaire items forty-six and forty-seven: 

1 . The responses will be recorded verbatim and categorized per employee group. 

2. The responses will be analyzed for recurrent themes. If themes develop, those themes 
appearing three or more times will be recorded in a table that will indicate the number of 
responses representing the particular themes. The table will also indicate the employee group 
or employee groups from which the responses came. 

Based on the review of the related literature and the recommendations of three research 
experts, the Employee Development subcommittee members developed a procedure for the 
future administration of the questionnaire. The population to be surveyed was identified. The 
Employee Deveiocfnent Committee members decided to survey a!! full-time and part-time 
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Governing Board Approved Paradise Valley Community College Center employees: One from the 
Crafts employee group, forty from the RFP-Residential Faculty employee group, eighteen 
from the MATP-Management/Adminlstrative/Technical Personnel employee group, twelve 
from the M&O-Maintenanco and Operations employee group, and forty-three from the 
PSA-Professlonal Staff employee group (114 employees total). 

Based on the review of related literature and on the recommendations of three research 
experts, a specific procedure for dissemination and collection of the completed questionnaires 
was developed as follows: 

1 . The Employee Development Committee members developed a formal letter to accompany 
the questionnaire (Appendix D). 

2. The Employee Development Committee members decided that they would personally 
invite employees to participate In the survey. Tliis will allow ttie oommittee members to 
reiterate the purposes of the questionnaire, to clarify that the questionnaire is to be filled out 
anonymously, to explain the directions for filling out the questionnaire, to explain that the 
questionnaire should be returned to the project director through campus mail, and to Inform 
that a summary of the findings from the survey will be made available to all employees. Since 
the Crafts and M&O-Maintenance and Operations work different shifts than do the Employee 
Development Committee members, the supervisor of these personnel will be asked to serve in 
the place of an Employee Development Committee member. 

3. Two weeks before the dissemination of the questionnaire, a nottoe will appear in the 
college bulletin announcing the purposes of the survey, the date that it will be disseminated, 
and a promise that the results of the questionnaire will be made available to all employees. 

4. One week before the dissemination of the questionnaire, the Provost will send an 

f 

electronic message to all employees. The message would again reiterate the purposes of the 
survey, include an announcement regarding the date of dissemination of the questionnaire. 
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include a promise to malce available a summary of the findings from the survey, and extend a 
personal invitatton to participate. Those employees who do not receive electronic messages will 
be sent a paper copy of the message. 

5. Employees will be given a deadline of one week to return the survey. One working day 
after the deadline, an electronic message will be sent to al! employees. The message will 
announce that it !s still possible to return the questionnaire through campus mail to the project 
director, and if anyone needs a copy of the questionnaire, a copy can be picked up at the 
information desk in the Student and Community Services building. 

Based on the review of related literature and on the recommendations of three research 
experts, the questionnaire was field tested. Two representatives from three employee groups 
(RFP-Residential Faculty, MATP-Management/Administrative/Technical Personnel, and 
PSA-Professional Staff (six total) filled out the questionnaire and provid^^d feedback regarding 
their understanding of the purpose of the questionnaire and the directions for filling out the 
questionnaire. The survey took fifteen minutes on the average to complete. The respondents 
understood the directions and filled out the survey accurately. 

In summary, as a result of this study, personnel of Paradise Valley Community College 
Center were given the opportunity to be better informed regarding the Employee Development 
Program at the college and were given a voice in developing the questionnaire that will 
eventually be used to evaluate the activities of the Employee Development Program. Moreover, 
the Provost of Paradise Valley Community College Center was provided with a questionnaire that 
can be revised yearly to evaluate the effectiveness of employees' participation in the college's 
employee development activities in terms of meeting one or more of the four goals of the 
Employee Development Program and to generate suggestions for new activities, lastly, the 
Provost was given procedures for the future administration and analysis of the questionnaire. 
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DISCUSSION AND RECOMMENDATIONS 
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A clear need for the evaluatton of the activities of the Emptoyee Development Program has 
been demonstrated. Based on the results of this study, the Provost of Paradise Valley 
Community College Center should charge the Employee Development Committee members witn 
the task of administering and analyzing the questionnaire developed for this study. Based on the 
analysis of the data generated from the future administration of the questionnaire, the Employee 
Development Committee should make recommendations for the improvement of the Employee 
Development Program. 

If the questionnaire Is Implemented, the following benefits are possible: 

1 . Given that employees will have a greater voice in shaping the activities of the Employee 
Development program, they may come to feel a greater ownership of the program and to 
perceive their professional and personal growth as a shared responsibility. 

2. With greater input from those affected by the program. It Is likely that the program will 
be improved for the better. 

3. Based on an analysis of the data generated from the questionnaire, the Employee 
Development Committee will be provided with a wholistic assessment of the effectiveness of the 
activities of the Emptoyee Development Program. Based on this assessment, problem areas 
can be Identified and follow-up studies can be conducted with the goal of redesigning or 
eliminating some of the activities. 

4. The Employee Development Committee will be provided with suggestions for additional 
activities that have the potential for aiding participants in becoming more effective personally 
and professionally. 

5. Employees will perceive that the success of the Employee Development Program is 
Important to the Provost of the college. Based on this concern, employees may perceive that the 
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their personal and professional growth Is of concern to the Provost. This perception could have 
the ultimate effect of bolstering the morale, motivation, and productivity of employees of 
Paradise Valley Community College Center. 

Furthermore, the Employee Development Committee members shoulc choose an appropriate 
week In the semester to administer the question, .aire. For example, the weeks of mid-terms and 
finals are busy times for faculty. Administering the questionnaire during those weeks could be 
perceived as Insensitive and thus could adversely affect the return rate. Lastly, the Provost 
should Invite a member of the M&O-Malntenance and Operations employee group to join the 
Employee Development Committee. In this way, the Employee Development Committee would be 
representative of the four major employee groups on campus. 

As noted earlier In this study, effective Employee Development Programs are important to 
the personal and professtonal success of employees of instituttons of higher education. People 
who are fn the business of teaching and learning should be constantly learning themselves. 
Instituttonal practices should support these learning efforts. 
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EMPLOYEE DEVELOPMENT 



rOUR GENERAL GOALS 



1. To give employees an opportunity to understand the mission 
of the institution and their role. 

All Employee Meeting (called at the start of each semester) 

North Central Accreditation Committees 

Employee group retreat 

Provost's Open Door Policy 

Immediate Supervisor's Open Doc/ Pclicy 

Don Creamer's Presentation on Student Development 

John Roueche'r, Presentation on Student Development 

Employee Reorientation 

New Employee Orientation 

Collegial Support Partnership Program 

2. To help employees improve their Job performance in terms of 
e. ectiveness, efficiency, and personal satisfaction. 

North Central Accreditation Committees 
Empioyeo Qroup Retreat 
Team Building (Management Devetopment) 
Campus Committees 

Johri Avianantos' Presentation on Goal Setting 

Jim Kern's Presentation on WeMness/Self-esteem 

Cynthia Scoffs presen' >on on Wellness in the WorltPlace 

Brown Bag Sessions 

Wellness Quests 

Welliiess 3reai<out Sessions 

Quiet Room 

S ' Care Center 

New Employee Support Program (Transition Aid) 

Employee Wellness Support Program (Individualized Prtjram) 

On-Campus Conferences/Seminars 

Off-Campus Conferences/Seminar ^ 

On-Campus A-1 Training 

MacUcense Training 

Faculty Innovation Series 

Collegial Support Partnership Program 
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3. To give employees opportunities for professional and personal 
development. 

Individual Employed Development Plan 

John Avianantos' Presentation on Goal Setting 

Brown Bag Sessions 

On-Campus Conferences/Seminars 

Off-Campus Conferences/Seminars 

Weight Watchers 

Faculty Forums 

4. To recognize and reward employees for their personal and 
professional contributions to the institution on a regular and 
continuing basis. 

Registration Celebrations 
(Holiday Celebration 
Secret Santa 

Employee Recognition/End of the ademic Year Ceiebretion 
IHonoring Retiring Employees 
Monthly Potluclts 
Appreciation Cards 

Star Workout Program (Employee Locker Privileges) 
Wellness Week 
Employee Picnics 
Employee Softball 
Emptoyee Golf Toumament 
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APPENDIX B 

EVALUATION OF THE EFFECTIVENESS OF EMPLOYEE DEVELOPMENT 
ACTIVITIES AT PARADISE VALLEY COMMUNITY COLLEGE CENTER 




Evaluation of the Effectiveness of Employee Development 
Activities at Paradise Valley Community College Center 

The purpose of this questionnaire Is to evaluate the effectiveness of PCCC's employee 
development s^th/ltles In helping you achieve the four goals of the Employee Development 
Program. 

The Emf^oyee Development Committee memljers will analyze the result of this questionnaire 
and use this Informatbn to Improve the employee development activities to better help you to 
achieve the goals of employee development. 

1 . In order to compare the perceptions of the different employee groups, piease indicate the 
employee group in which you are a member on the "General Purpose Data Sheet 1" using the 
following code: 

A - Crafts &M&0; B - Faculty: 0 - MAPI; D - PSA 

GOAL NUMBER ONE: To give employees an opportunity to 
understand the mission of the institution and their role. 

The foltowing employee development activities were designed to provide you with an of^rtunlty 
to achieve goal number ono of Vie Employee Oevebpment Program. 

How effectively did your participation (from Fall semester 1988 to the present) in the 
activities listed below help you to achieve employee development goal nurr,ber one? 
Pleaso 8.nswer on the "General Purpose Data Sheet I" using the following code: 

A - Very Effet :ve; B - Effective: C - Neutral: D - ineffective; 

E - Very ineffective; !f vom did not participate in an activity, please leave 

that item blank. 

2. A'l Employee Meeting (called at the stirt of each semester) 

3. North Central Accrertitation Committees 

4. Employee group retreat 

5. Provosfs Open Door Policy 

6. Immediate Supervisor's Open Door Policy 

7. Don Creamer's Presentetion on Student Development 

8. John Roueche's Presentation on Student Development 

9. Employee Rejrientation 

1 0 . New Employee Orientation 
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GOAL NUMBER TWO: To help employees improve their job 
performance in terms of effectiveness, efficiency, and 
personai satisfaction. 

The following employee devebpment activities were designed to provide you with an opportunity 
to achieve goal number two of the Employee Development Program. 

How effectively did your participation (from Fall semester 1988 to the present) in the 
activities listed t)eiow help you to achieve employee development goal number two? 
Please answer on the "General Purpose Data Sheet r using the following code: 

A - Very Effective; B - Elective; C - Neutral; D - Ineffective; 

E « Very ineffective; If ycu did not participate in an activity, please leave 

that item blank. 

1 1 . North Central Accreditation Committees 

1 2 . Employee Group Retreat 

1 3 . Campus Committees 

1 4 . John Avianantos' Presentation on Goal Setting 

15. Jim Kern's Presentation on Wellness/Self-esteern 

1 6. Cynthia Scott's presentation on Wellness in the Workplace 
7 . Brown Bag Sessions 

18. Wellness Quests 

1 9. Wellness Breakout Sessions 

20. Quiet Room 

21. Self-Care Center 

22. Employee Wellness Support Program (Individualized Program) 

23. On-Campus Conferences/Seminars 

24. Off-Campus Conferences/deminars 

25. On-Cempus A-1 Training 

26. MacLicense Training 

27. Faculty Innovatton Series 
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GOAL NUMBER THREE: To give employees opportunities for 
professional and personal development. 

The following employee development activities were designed to provide you with an of^rtunity 
to achieve goal number three of the Employee Development Program. 

How effectively did your participation (from Fall semester 1988 to the present) in the 
activities listed below help you to achieve employee development goal number three? 
Please enswer on the "General Purpose Data Sheet 1" using the following code: 

A - Very Effective: B - Effective: C - Neutral: D - Ineffective; 

E - Very ineffective: If you did not participate in an activity, please leave 

that item blank. 

28. Individual Employee Development Ran 

29. John Avianantos' Presentatbn on Goal Setting 

30. Brown Bag Sessions 

31 . On-Campus Conferences/Seminars 

32. Off-Campus Conferences/Seminars 

33. Weight Watchers 
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GOAL NUMBER FOUR: To recognize and reward employees for 
their personal and professional contributions to the 
institution on a regular and continuing basis. 

The following employee development activities were designed to provide you witti an opportunity 
to achieve goal number four of the Emptoyee Development Program. 

How effectively did your participation (from Fall semester 1988 to the present) in the 
activities listed below help you to achieve employee development goal number four? 
Please answer on the "General Purpose Data Sheet I" using the following code: 

A - Very Effective; B - Effective; C - Neutral; D - Ineffective; 

E - Very Ineffective; If you did not participate In an activity, please leave 

that item blank. 

34. Registration Celebrations 

35. Holiday Celebration 

36 . Employee Recognition/End of the Academic Year Celebration 

37. Honoring Retiring Employees 

38. Monthly Potlucl^s 

39. Appreciation Cards 

40. Star Worl^out Program (Employee Loclter Privileges) 

41. Wellness Weelt 

42. Employee Picnics 

43. Employee Softball 

4 4 , Employee Golf Tournament 
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45 . Of the four Employee Development Goals listed below, which one to you needs more 

attention? Pleaso answer on the "General Purpose Data Sheet r using the following code: 

A - Goal Number One: To give employees an opportunity to understand the mission of the 
institution and their role. 

B - Goal Number Two: To help employees improve their Job performance In terms of 
effectiveness, efficiency, and personal satisfaction. 

C ■ Goal Number Tnree: To give employees opportunities for professional and 
personal development. 

D - Goal Number Four: To recognize and reward employees for their personal and 
professional contributions to the Institution on a regular and continuing basis. 



46 . Please comment on suggestions that you have for additional employee development 
activities. Please write your suggestions in the "Write-in Area 1" on the "General 
Purpose Data Sheet 1 ." 



47. Please comment on suggestions you have for improving the design and the administration 
of this questionnaire. Please write your suggestions in the "Write-In Area 2" on the 
"General Purpose Data Sheet 1." 



48 



ERIC 



APPENDIX C 
GENERAL PURPOSE DATASHEET I 
FORM NUMBER 19543 
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PARADISE VALLEY COMMUNITY COLLEGE <"£NTER 

MEMORANDUM 



Date: 

To: (Name of employee) 

From: The Employee Development Committee (John Cordova, Marilyn Cristiano, S'^iriey 
Green, John Henderson, Ernie Ijira, Jerome Baxter, Sue Shuman, Cheryl Kubasch, 
Karen Watl^ins, Alexis Thieike, and Loretta Mondragon) 



Tue Employes Development Committee Invites you to complete anonymously the enclosed 
Evaluation of the Effectiveness of Employee Development Activities at Paradise Valley 
Community College Center questionnaire. 

All full-Vme and part-time Governing Board Approved PVCCC employees are being invited to 
: 4rticipate in this survey. 

The purpose of the questionnaire is to evaluate .he effective less of PVCCC's employee 
development activities in helping you achieve the four goals of the Employee DeveloDrr,qnt 
Program. The results will be used by the Employee Development Committee to better help you 
to achievii the goals of employee development. 

DEADLINE: (DATE) 

DIRECTIONS FOR FILLING OUT THE SURVEY: 

1 . For items 1 through 45, please respond on the enclosed "General Purpose Data Sheet I." 
Please use a number 2 pencil. If you have not participated in an activity, 
please leave that item blank. 

2. For items 46 and 47, please respond in t!ie "Write-In Area i" and "write-in Area 2" 
respectively. 

PLEASE RETURN THE SURVEY TO {Name of Project Directoi) 
THROUGH CAMPUS MAIL BY (DATE) 

Thank you for helping the Employee Development Committee to improve 
tne Employee Development Program! A summary of the findings from this 
survey will be made available to all employees. 

Sincerely, 

(Signature) 

John Cordova, Provost 



ERIC 
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